An Empirical Evaluation of Testing Efficiency during Non-operational Testing

In this paper we discuss the question of growth of testing efficiency during non-operational testing. "Learning" is said to occur if there is an improvement in the efficiency of testing during a testing phase. Our work indicates that in practice very little actual "learning" may take place during non-operational testing, particularly if test duration and testsuite size is subject to a priori constraints. In fact, some of our results show that "learning" may be an artifact of the program structure. We discuss the testing process that we believe may operate in an non-operational testing phase, and we develop a fault detection model based on that process. Finally we present and discuss some early empirical results on the growth of testing efficiency.